{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# MNIST classfification using multinomial logistic + L1\n",
    "# 基于L1惩罚的多项式Logistic回归模型的MINST手写数字数据集分类\n",
    "\n",
    "Here we fit a multinomial logistic regression with L1 penalty on a subset of the MNIST digits classification task. We use the SAGA algorithm for this purpose: this a solver that is fast when the number of samples is significantly larger than the number of features and is able to finely optimize non-smooth objective functions which is the case with the l1-penalty. Test accuracy reaches > 0.8, while weight vectors remains *sparse* and therefore more easily *interpretable*.\n",
    "\n",
    "Note that this accuracy of this l1-penalized linear model is significantly below what can be reached by an l2-penalized linear model or a non-linear multi-layer perceptron model on this dataset.\n",
    "\n",
    "采用L1惩罚的多项式Logistic回归对MNIST手写数字数据集的一个子集进行分类，使用SAGA算法——这是一个快速的求解器，当样本数量明显大于特征数量时，在L1惩罚下，能够精准地优化非光滑目标函数。测试精度>0.8时，而权重向量保持*稀疏性*，因此更容易*分辨*手写数字。\n",
    "\n",
    "值得注意的是，比起此数据集上的L2惩罚的线性模型或非线性多层感知器模型，L1惩罚的线性模型其精度更低。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Automatically created module for IPython interactive environment\n"
     ]
    }
   ],
   "source": [
    "print(__doc__)\n",
    "\n",
    "# Author: Arthur Mensch <arthur.mensch@m4x.org>\n",
    "# License: BSD 3 clause\n",
    "\n",
    "import time\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "\n",
    "from sklearn.datasets import fetch_openml\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "from sklearn.utils import check_random_state"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Turn down for faster convergence\n",
    "# 设置少量样本数，加快模型训练运行速度\n",
    "t0 = time.process_time()\n",
    "train_samples = 5000\n",
    "\n",
    "# Load data from https://www.openml.org/d/554\n",
    "# 加载数据集\n",
    "X, y = fetch_openml('mnist_784', version=1, return_X_y=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "# seed=0，固定生成一组随机的序列，打乱数据集\n",
    "random_state = check_random_state(0)\n",
    "permutation = random_state.permutation(X.shape[0])\n",
    "\n",
    "# 得到打乱之后的训练集\n",
    "X = X[permutation]\n",
    "y = y[permutation]\n",
    "X = X.reshape((X.shape[0], -1))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "L1惩罚下的模型稀疏度: 81.81%\n",
      "L1惩罚下的模型性能: 0.8296\n"
     ]
    }
   ],
   "source": [
    "# 交叉校验，训练集5000条，测试集10000条\n",
    "X_train, X_test, y_train, y_test = train_test_split(\n",
    "    X, y, train_size=train_samples, test_size=10000)\n",
    "\n",
    "# 对训练集和测试集进行标准化处理\n",
    "scaler = StandardScaler()\n",
    "X_train = scaler.fit_transform(X_train)\n",
    "X_test = scaler.transform(X_test)\n",
    "\n",
    "# Turn up tolerance for faster convergence\n",
    "# 设置收敛容差，加快模型训练运行速度\n",
    "clf = LogisticRegression(C=50. / train_samples,\n",
    "                         multi_class='multinomial',\n",
    "                         penalty='l1', solver='saga', tol=0.1)\n",
    "\n",
    "# 训练模型\n",
    "clf.fit(X_train, y_train)\n",
    "# 计算稀疏度\n",
    "sparsity = np.mean(clf.coef_ == 0) * 100\n",
    "# 得到模型性能\n",
    "score = clf.score(X_test, y_test)\n",
    "\n",
    "# print(\"Sparsity with L1 penalty: %.2f%%\" % sparsity)\n",
    "# print(\"Test score with L1 penalty: %.4f\" % score)\n",
    "print(\"L1惩罚下的模型稀疏度: %.2f%%\" % sparsity)\n",
    "print(\"L1惩罚下的模型性能: %.4f\" % score)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "本例运行时间：37.128 s\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAkMAAAFCCAYAAADyjdmjAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzt3XmYZFlZ5/HfG5H7WlmVtVdlVXctvRXdTdPQQLM00K0I2ogyM4ADwwzDo8+gzMCA4wIzqI/TOjrKjILOAyKIAqIgioKAYNtNCy3QC9ANvdfStWctuVTlGnHmj4i695xbGVmRmREZGXm+n+epp86Nc/PeG3Hi3jx53rOYc04AAACxyjX6AgAAABqJyhAAAIgalSEAABA1KkMAACBqVIYAAEDUqAwBAICoURkCaszM3mtmf1rH4z9kZreU02Zmf2xmZ8zsX8zshWb2SB3OOWRm42aWr/Wxm4WZvdrMDpU/h2c2+noA1A6VIWARzOz1Zvat8i/Go2b2BTN7wXKc2zl3jXPuzvLmCyTdJmmbc+45zrm7nXNXLPUcZrbfzG71znnQOdfjnCss9djLwcycme2u8WF/W9LPlj+H+2t8bAANRGUIWCAze4ek90n6n5I2ShqS9AFJr2rA5eyQtN85d64B516VzKylQtYOSQ8t8pjRtqgBzYDKELAAZtYv6VclvdU59xnn3Dnn3Ixz7nPOuXdV+Jm/MLNjZjZiZneZ2TVe3ivM7GEzGzOzw2b2zvLrg2b2t2Z21sxOm9ndZpYr5+03s1vN7M2SPiTpeeUWql8xs1vM7Gnv+NvN7DNmdtLMTpnZ75df32VmXy2/Nmxmf2Zma8p5H1Opgve58nF/3sx2lltbWsr7bDGzvylf2+Nm9hbvnO81s0+Z2Z+U39dDZnZjhc/mD83stzOv/XW5wnnhPJ8uX/9TZvY2b7+8mf2SmT1RPs+3y+/3rvIuD5av/9+U939L+VpPl699i3csZ2ZvNbPHJD2WuZ52MxuXlC8f84ny61eZ2Z3lMnrIzG73fuYjZvYHZvZ5Mzsn6SVzvX8AK4Rzjn/841+V/yS9XNKspJZ59nmvpD/1tv+DpF5J7Sq1KD3g5R2V9MJyekDSDeX0HZL+UFJr+d8LJVk5b7+kW8vpN0n6mne8WyQ9XU7nJT0o6XcldUvqkPSCct5ulcJr7ZLWS7pL0vu84yTnKG/vlOQuvG9J/6RSa1iHpOslnZT0Mu/9T0p6Rfka7pD0jQqf1YskHfLe24CkCUlbVPpj7duS/rukNkmXS3pS0g+X932XpO9KukKSSbpO0rpynpO02zvPSyUNS7qh/J5/T9JdXr6T9GVJayV1VrjW5JjlMnlc0i+Vr+2lksYkXVHO/4ikEUk3l99HR6O/u/zjH/8q/6NlCFiYdZKGnXOz1f6Ac+7Dzrkx59yUShWF68otTJI0I+lqM+tzzp1xzt3nvb5Z0g5Xanm62zm30IUEn6NSpeJdrtSCNemc+1r5mh53zn3ZOTflnDsp6Xckvbiag5rZdpX6Kv238jEfUKmF6g3ebl9zzn3elfoYfUylispc7lapkvHC8vZrJH3dOXdE0rMlrXfO/apzbto596SkD0p6bXnf/yjp3c65R1zJg865UxXO81OSPuycu69cDr+oUovaTm+fO5xzp51zE1V8DM+V1CPpN8rX9lVJfyvpdd4+f+2cu8c5V3TOTVZxTAANQmUIWJhTkgbn6VcSKIdyfqMcyhlVqcVFkgbL//+kSi0oB8zsn8zseeXXf0ullocvmdmTZvYLi7jW7ZIOzFVxM7MNZvbJcmhuVNKfetd0KVsknXbOjXmvHZC01ds+5qXPS+qY6zMrV/A+qbQS8XpJf1ZO75C0pRyGOmtmZ1Vqidnovb8nFnDNB7zzjqtUlv41H6ryWBeOd8g5V/Rey34GCzkegAaiMgQszNdVCgH9eJX7v16ljtW3SupXKdwklcI6cs590zn3KkkbJH1W0qfKr4855/6rc+5yST8m6R1m9rIFXushSUMVKm53qNQic61zrk/Sv71wTWXztUIdkbTWzHq914YkHV7g9V3wCUmvMbMdkm6S9Gnv+p9yzq3x/vU6517h5e+q8hxHVKpcSZLMrFulVj7/mhfS8nZE0vYL/bjKsp/BQlvyADQIlSFgAZxzIyr1YXm/mf24mXWZWauZ/YiZ/a85fqRX0pRKrRBdKo1AkySZWZuZ/ZSZ9TvnZiSNSiqU837UzHabmXmvL3RY+7+o1CfpN8ys28w6zOxm77rGJZ01s60q9b/xHVepj85cn8EhSf8s6Y7yMa+V9GalLToL4krD1E+qFGr7onPurHf9o2b238yss9zKts/Mnl3O/5CkXzOzPVZyrZmtq3D9H5f0783sejNrV6kc7nXO7V/MNUu6V9I5ST9fLv9bVKq0fnKRxwPQQFSGgAVyzv2OpHdIerdKv8QPSfpZlVp2sv5EpfDJYUkPS/pGJv8NkvaXQ1U/o1ILjSTtkfQPKlVYvi7pAy6dW6ja6yyo9At6t6SDkp6W9G/K2b+iUmfiEUl/J+kzmR+/Q9K7y+Gpd85x+Nep1Mp1RNJfSfofzrkvL+T6Mj6hUuvZx+e4/uslPaVSB+gPqdTCJpX6OX1K0pdUqjD+kaTOct57JX20fP3/2jn3FUnvUanV6ahKLUoX+h4tmHNuWtLtkn6kfF0fkPRG59wPKv2Mmb3HzD7nbX/JzH6+nM6XR749r9LPA6ifCyM4AAAAokTLEAAAiBqVIQAAEDUqQwAAIGpUhgAAQNSoDAEAgKhRGQIAAFGjMgQAAKJGZQgAAESNyhAAAIgalSEAABA1KkMAACBqVIYAAEDUqAwBAICoURkCAABRozIEAACiRmUIAABEjcoQAACIGpUhAAAQNSpDAAAgalSGAABA1KgMAQCAqFEZAgAAUaMyBAAAokZlCAAARI3KEAAAiBqVIQAAEDUqQwAAIGpUhgAAQNSoDAEAgKhRGQIAAFGjMgQAAKJGZQgAAESNyhAAAIgalSEAABA1KkMAACBqVIYAAEDUqAwBAICoURkCAABRa1nIzoODg25oaKhe14JLOHjwoIaHh60Wx6IsG6uWZSlRno3Gvbl6UJary/333z/snFt/qf0WVBkaGhrSPffcs/irwpLcfPPNNTsWZdlYtSxLifJsNO7N1YOyXF26uroOVLMfYTIAABC1BbUMAcByG58pBts9rfwNB6C2eKoAAICoURkCAABRozIEAACiRp8hACsafYQA1BtPGQAAEDUqQwAAIGqEybCiVDvtq6vrVWApvntyomJeV2t+ztfb8mHJ+3+lPXo6PN4zNnQn6cHOuY8HAAtByxAAAIgalSEAABA1KkMAACBq9BlCw/m9RegL1HzOZZbLmCmkpdjREv69dXRsKkmfODedpNsz+41PzSbpk95+krS2o9VLdwZ5uZqsNY4Lit4NeXaqEOT55b6pOy0TZkJoblacrZjncqu3ysDXFgAARI3KEAAAiNrqbfNaoOlCGKDJDvVF7fDJNj8/fHLyfBg+WduZhkyyYavWfPr3V97SzMlCGGrb1tuRpJ+xMTzIpp7Kx8elZcvLNz4T5u0/O5mknzx9Psg74YU8923qTdJ71nUH++1d256kF1JeTLOxfKyQhqJz0+lUFrmx4xX3m9l0dZDnbOltK5OZ38Mdy/h7mJYhAAAQNSpDAAAgalSGAABA1KLuM3RmMo2Pj02HfRa87gza3tsqLI0f+bXJ0SAvd/5MknatHUFeoXdjPS8LNbCxO3yMdLZUF+f3f66vjb/LFuqxM+GUAzv60ufUxGzY98KfuSDbH/Lhk2lfoMdOnwvyhsfTc4xNhkOuf2jv+iR9xWA6xcFFS6ssstsHfYGWT378ZJJ2j9+XpA//3eeD/aZH0+/K9lf9UJDXcu0tSXq2f+uiriP7FBj3pm/oqfOcDTyBAABA1KgMAQCAqEUdJutrT1e8zmfacidm0+a50UwIjSb9pclNjATbdvZoknYjp4K8ti27kvT0xivre2Gomn+7VBsWy+I+Wpo9A23Btj8qOZ8Lg0z+1CGtmTDWnnVpiOvGLT1BHrNJr04X3bGHH0mS+//sU0n6qX94Mtht+Ew67N7y4Zdj55U3Lfm6siHW5Zzihq86AACIGpUhAAAQNSpDAAAgaqu+z5C/oHY2/u2HI7P9F/zt05PhFPX+dlfmoMs5ffhKdtGn4LyCyIdfO2tJ+z64yXDKfxXC4cMA5uY/ei4ahjzP7CA98+T59zFD3VexbVclye2vuCVJu8wyOTu87e0/8aNB3sz63fW5tmVCyxAAAIgalSEAABC1VRkm82dfXeywX9/ajnyw7YfJZouZxmPCZJIublI3b0Vjlw+HBKsl/RpaWzgDtWvtmvP42Rl2H/Jm0b1xc3d294py0+cr5hXb5j43sJoEs8Nn74f7v5Aki2dOBlntVz8nSU8P3VDVuYYnwi4Hg535CnuinrLP59m+zUk6d8tPJenLt1wW7GctaUx1Zu8L63FpDUPLEAAAiBqVIQAAEDUqQwAAIGqros/QTDj6r+7ddvw+RMfOhSs5T3p9WbL9ifKWXtj6rrhi5f4n4fKZsbyz6Wdo7Zk+Qy3tcx7vjjvDaeK/8E9PJelv/tqtFa+jZeRwsG0zU0m6sGZLxZ9DZbmp8WA7P3YiSdt4uLyKOnuTZMHrp1Do7K/PxeEi2cdjy6n9SXrq7k8HeUe+9p0kvfaqHUFe7rZ9VZ3vwePpEg5Hx6eCvJfvWlPVMbLXzDD/+im2pM9g235NkDfbu3G5L2fZ0DIEAACiRmUIAABEbVWEyS5eWXnhcTLLzHR80fDvCs5lYnSPnUqHps5kZu/saU8/7pfs6FvoJa4aVgyH1xbPjybpwvDRIC+/Ze+cx/jYn9wZbPdt2lbxfH4YJzdyPDx3f9rs6zcPI5SdgiA/ciRJFw9+P8ibOvRokh47GH7efXsvT9Ktz/mRJG3t4XQILrcqHk0rhv9EzI+G91jx6R8kaVcI780ttzwr/bkffVuQ54eqRqfTZ937v3Eo2O8HR9L7+z+96HKheRRXcVgsi5YhAAAQNSpDAAAgalSGAABA1KIKzLecPhBs57xhv1M/+Fa47/qtSbrwzFdWPGZPZrX7ew+cSdJDa8PlHJ4enUzS128M+0gMdMQz1N5mwv4nhVPH0vTo6SCvpTAz5zFOP35fsP2a11aeGj5/4P703N1hX61CRDHxpWg5czDYLhx5IklP7w/7DJ07lpbh5KmRIK938lySzk+cTdKuvSc8fve6xV8s5lVs7w22WzbtTNKd68PpJWa2XFvxOH5/yZ/7zENJ+smnzgT7veTZ6bN0Y3d1fTGlxfT8RMHryDU+HfZZnfT6sG7sqvJXf6YvrarsS9uMaBkCAABRozIEAACi1jRhsuwq5dWuRt9++MEkPXPw0SBv4sBjSfr4tx4J8jbeeEWS7rjyBUGeP1tuZ0tYn9w20Jmkiy685rWd6czLPW3xhMUuUghn7dZM2hQ7tv9IkNX6krRJ35/Qe+3ucJXsn37ezoqnm3r4X5J0+7U3h5nG3wOVmPOa2bNTT8ym4UvLh9/lznVpKLLvij1BXtveZ6bH8GZkl8tMI4+aCp5EmZDk7MBQul9r5eklsrM+/6Y36/vlG9Jj/swLwpXOr9mQdhfoqPfyAJE5NBZ2Izgylt6n+8+E3RFu3Jr+3toY9uCoqNopZiRp3Aub9lw8382K13xXDAAAUENUhgAAQNRWdJjMbwIc7Kz+UvPjJ5P09JPfS9KTB58K9hvdn87EOj0WNim2bU0XJZxvEcnpQth4fLk3gqy/PbzmrtY0nNCErYg1Y5nwYXEy/ewtH34w/mivo973YfeNVwf77R5Im3NbTz4e5B1/NB0JtTETJrOZdISfa6uy7TgSzgshzq4LZw7O9axP0m27nxXkuY40TJa9dzJjU7ACLPZ7/z9etitJ1yP6xWKsc/N/L37iwXA28YeeTkdvbugLF7nu70i7aWzqDn83dS/iF9LhTIhuZCqdvby/PQydb+3NLM69AkX8KxkAAIDKEAAAiByVIQAAELWG9xnyZzE9di4ccj01u4gZMxWuyjzjzW48ceJssJ+/veH6cAhwy5U3pceY51yFTP+XK9al8fdsHLba6QBWO9caxrJbh9KV6dfuvDLI878RT51N+/ds3xTOousrPH5/sJ1rTePV1hIOFbUKM1wjVMwMx85ux+LMZLiq+6qYOb7o3WW5ys/Z7NOr1v2EiplOQjkel5Iu/lyeOjORpEfOh73wJqbT7+eB4bAf7MNrxpJ0dmqX52xJ7+e2eQr2+Pn0u3Lv4dEg7/HhdHb5269qvpn9aRkCAABRozIEAACituxhsuGJsJnZD4Xt95r/JGlrX+XZUOc1PZUmR9Omu+mxc8Fua/ZuT9IDt/1okDezfndVp8plGo/XdjF8/lKKbeEitfkNO5P07LqdquTup9IFQK8bWlNxv8KZE8F232Wbk3R2tmRmPq6vyczUExNeWLzDm719vhBy9hiNnMV4dhV+XfypSGxyLJOZhpULPYNBVq6YPstz59JFr92J/eExtl2VHqNvc5Dll+xUppz9odv+1BmxyYYLn7s17SLQ3hI+zw4PpV0JNveE3RF8WzPD7qe9WFyLd8KzU+Hv63sOpkP3/bBY9uc2dDe8B86C8esaAABEjcoQAACIGpUhAAAQtYYH9mbmicGPTKXD+FpyleOfF+lMY6qdl6VTxrdv2hTs1rYzjWXP7H5+9cf39LeH9Un6CV1a0VuuYa7tSh47lvZn+O8/fEXF/dp2Xxtsu+k0jl5s7QzzWhbwvVoF5hu+nL0XT02k9197pp9O3vvBbJ+Ggneck+fD6TJmvQto9Y6ZmaFCh0bTMrt6fbhcRMcCptlYjOwUHzPeNW9vgmUFFsrvw9dyNlzeoTia9gXS2DeDvMK5dGj1zKTXf6QYfpE6BjakP5PpM+SzzPdoYrYw946R84e+37Ql7H+p7PYSrc1MHXGdN6XJDZvD5/bO/ua+N/jVDQAAokZlCAAARG3Zw2SDnWGzm7/q+/mZsFl02Jtd88p1YXgjG57yzQymK2znvVWzc5nZVWe8lbcXa77ZOrF0J8+n34mb96RDe+drknXrdwbbubF06PBFYbJ8wyPFy2q+WX0fPB4Olf3u8fEkfePWsEl8R38aXsyG3vafTae2+MoTw0Fepzfz7dCatCx6MzPibvem1Rhor/0sz9nh+g+fTKf1uPfpcKb6269Y+nNiJbloNXgvTF3szbzX8fSzcFPh1CfnDxxI0pZPn8ddu8LZ/Atr0ilM5luJfmo2zG3N8bf6SrNrzeqd4oBvGwAAiBqVIQAAEDUqQwAAIGoN7zDR5/X9GegM+4G0e9P1n8gM0e1vrxy7dN4U8rPzDOXEytfZmnZyeeXewXn29GSX2GhJv+Yus2q9/12J0eh0+lll77+i8/vzhZ/pWW/19uHzM0Hek2fS1bIHe8LPd3t/2k9o10Ca3twTnrsWU1SMTIXX7PdlGpkKr/k73rQN67rCa86v0OXTJzJ9bOZb0qRahYHtwXauPV3NvNUbIi9Jay7fl25491ihf2uwX9Hrtzmf0emwz+iajtr3FVvtcpPpdAf5sXBZIv+5WFg7FGQVWxa59NUqQssQAACIGpUhAAAQtYaHyXq89vArB8OZZke8FXNPjE8HeWu84bbru2hOXa3870dPlbET5zXtS5JmvCHBuYZ/5VcUPwLU11Z5ttmsJ85MVMzbszadBbcvE+q4rL++YUl/yPzhsfCZMePNjDyWCcls6k2nCrhpWxjW2dTAFbgfPB5+ztdtTEOLpyfCrgOd89wf2ZmEfX6wLRtoK3YNzJleiPmG0/uzfWe/f/NNn4KS3NR4sN1yen+SLo6eDnf2vv/Zb4PbsDdN1+rimgzfNgAAEDUqQwAAIGpUhgAAQNRWVAeK7FIdHd5Q0eyU/0e8/gDThfBtbF2FK0ujeoXudeELltb5ix2V+8HUit/vYqXH3/1+WCOT4XBzf8WKbN5abxj+jv5wWG4j+/AVvAdFLrsMumfXQNg/sbst/RyaZWX67HPu4eHJivtOeFMjzPd8zH5f5xusv5jv9qGxzPfI68vUXYv5FGKTmUbEeUPkc31rw32956Cz8LM2b0i+6wiX3okF3z4AABA1KkMAACBqKypMluU34e/oC5t2j51LG3CHM7NT+5Pl9njN39nhpSt0YlnUWMEbEpwtcr4CqWz4ZF1X+njIZ0JOtYho5MeOJ+ncxEiQV+xJV08vLGBItx9q2TXQHuRNzKbvz5+2Q5IGO1fmo9AfSn8p/tD63sww9YnZ9KGYnZnbL9pCpj9Ch7cKQHaGa39X/1mamaxczpvJvFlCkM2imA1pzTN1SG7Su8dmpzOZ6c81U5i/lmgZAgAAUaMyBAAAokZlCAAARG1lBsrn0JYP49VDXh+ioUx/Ij9mzWhN+GKKgS9VR37pParymb5AOa+fUOHAw0na9a4J9rNWr6/MIpeByN77rV7/wb625n8w+EuPSOFSJ9mpSA6Ppw/F/WengrxNXj+e2cwxJ2bT7clMn6GBCkt8XPzMpWfecim2dS0qD7QMAQCAyFEZAgAAUWuaMNlCEBprXtlVmK2QzlhbbE9XQ3f5+q5+jsWzYjrVRXbGb78MtX53kpw1btqFmi+MmZ02xJ+mpC0zvL2/PQ131SI0CjQjnkAAACBqVIYAAEDUqAwBAICorco+Q2hexfaeRl8ClsjNsySA6BvUEJWGwQOSZC6deiG7on0s4nzXAAAAZVSGAABA1AiTAQAQsVhDYz4+AQAAEDUqQwAAIGpUhgAAQNSoDAEAgKhRGQIAAFGjMgQAAKJmzrnqdzY7KelA/S4Hl7DDObe+FgeiLBuuZmUpUZ4rAPfm6kFZri5VleeCKkMAAACrDWEyAAAQNSpDAAAgak1TGTKzTWb2STN7wsweNrPPm9leM9tpZt+r0znbzezPzexxM7vXzHbW4zyxaVBZvsjM7jOzWTN7TT3OEasGlec7yuf6jpl9xcx21OM8sWlQWf6MmX3XzB4ws6+Z2dX1OE+MGlGe3rlfY2bOzG6s53lqpSkqQ2Zmkv5K0p3OuV3Ouasl/ZKkjXU+9ZslnXHO7Zb0u5J+s87nW/UaWJYHJb1J0sfrfJ6oNLA875d0o3PuWkl/Kel/1fl8q14Dy/LjzrlnOOeuV6kcf6fO54tCA8tTZtYr6W2S7q33uWqlKSpDkl4iacY594cXXnDOPeCcu9vfqVzbvbvcAnCfmT2//PpmM7ur/JfH98zshWaWN7OPlLe/a2Zvn+O8r5L00XL6LyW9rPwFw+I1pCydc/udc9+RVKz3G4xMo8rzH51z58ub35C0rY7vMRaNKstRb7NbEqN6aqNRvzcl6ddUqthO1uvN1VqzrFq/T9K3q9jvhKTbnHOTZrZH0ick3Sjp9ZK+6Jz7dTPLS+qSdL2krc65fZJkZmvmON5WSYckyTk3a2YjktZJGl7qG4pYo8oS9bESyvPNkr6w2DeARMPK0szeKukdktokvXTJ7wRSg8rTzJ4pabtz7m/N7J01ei911yyVoWq1Svp9M7teUkHS3vLr35T0YTNrlfRZ59wDZvakpMvN7Pck/Z2kL81xvLlagfirZXnUuizRWHUpTzP7tyo9uF9c16uHr+Zl6Zx7v6T3m9nrJb1b0r+r95tAomblaWY5lbqUvGm5Lr5WmiVM9pCkZ1Wx39slHZd0nUoPyDZJcs7dJelFkg5L+piZvdE5d6a8352S3irpQ3Mc72lJ2yXJzFok9Us6vZQ3goaVJeqjYeVpZrdK+mVJtzvnppb2NqCVcW9+UtKPL+bicZFGlGevSi1Sd5rZfknPlfQ31gSdqJulMvRVSe1m9pYLL5jZs80s+9dgv6SjzrmipDdIypf33SHphHPug5L+SNINZjYoKeec+7Sk90i6YY7z/o3Sv1BeI+mrjlkql6pRZYn6aEh5lpvi/59KFaETdXhfMWpUWe7xNl8p6bEavqeYLXt5OudGnHODzrmdzrmdKvXnu9059636vMXaaYowmXPOmdmrJb3PzH5BpU5Z+yX9l8yuH5D0aTP7V5L+UdK58uu3SHqXmc1IGpf0RpX6A/1xuVlPkn5xjlP/kUo14sdVahF6bc3eVKQaVZZm9myVRlYMSPoxM/sV59w1tXxvMWrgvflbknok/YWVxjQcdM7dXqv3FaMGluXPllv5ZiSdESGymmhgeTYlluMAAABRa5YwGQAAQF1QGQIAAFGjMgQAAKJGZQgAAESNyhAAAIgalSEAABA1KkMAACBqVIYAAEDUqAwBAICoURkCAABRozIEAACiRmUIAABEjcoQAACIGpUhAAAQNSpDAAAgalSGAABA1KgMAQCAqFEZAgAAUaMyBAAAokZlCAAARI3KEAAAiBqVIQAAEDUqQwAAIGpUhgAAQNSoDAEAgKhRGQIAAFGjMgQAAKJGZQgAAESNyhAAAIgalSEAABA1KkMAACBqVIYAAEDUqAwBAICoURkCAABRozIEAACiRmUIAABEjcoQAACIGpUhAAAQNSpDAAAgai0L2XlwcNANDQ3V61pwCQcPHtTw8LDV4liUZWPVsiwlyrPRuDdXD8pydbn//vuHnXPrL7XfgipDQ0NDuueeexZ/VViSm2++uWbHoiwbq5ZlKVGejca9uXpQlqtLV1fXgWr2I0wGAACiRmUIAABEjcoQAACIGpUhAAAQtQV1oAbq4Ymz00l6+PxMkDc+NZukT5xL9zs7Ge63pbcjSa/raq14ri89ciLY/slnbEnS2/vbgry1Hfn5LhsAsErQMgQAAKJGZQgAAESNMBkabteaNDyVDU0NTxSSdFdrmveFH4wG+3323kNJeuv67iDv2JmJJD1+djLIOzE6leZNzgZ5v/ljV6XH7K0cejszWQi2Cy5ND3YSagOAlY6WIQAAEDUqQwAAIGpUhgAAQNToM4QVZSDTZ8jf3jOQ9i163rae8Adv3VXV8Sf9Dj2S8paux9i6yD8NstcMAKtBdrVamz6fbhTDPpaurStN55qvakHLEAAAiBqVIQAAELXma8taYWaK4fbTY+ksybOZvBav6lnI5O0eCGc/Rn105LMNv1hOudnfOxWoAAAT3klEQVR0aoPc2MkkbTMTwX5WSO8jzUwHeWpPm+ML3euCrELP+hpcJWrp0Fg6W/xsMQxTt+fTh+LG7vDXEbdq/fj3oSTlRo+n6fFTSbo4eirYrzCSbhdOHQuP0bMmSefXbQry8uvTmf4LvRuDPNfZn56vpUONQssQAACIGpUhAAAQNSpDAAAgavQZWoRpb3j2906GfR2++GjaD+I7B88GeZ1t6RDszWvC2Oiv3bY7SeeIlWOV8vsEWGuazo2E/Q+K58fS9Llw6ZVcb9o3IZcP+9oVvT5DYe8U1NPEbPppnzwfDrken06XqxmbDpeuWddZeZmb4BiZzpk9i5wHw++ytNqfs/lzYX+f/MjhJO3awqlJCr0bkvTs2h1VHX++EijMs20uLMucd535/FSQV/D6E9UbLUMAACBqVIYAAEDUCJMtQps35vOGTV1B3s4125L0qWdsDvIePXUuSX/lByeCvJGptCGRGY2XJhiWLan10INJunAm/NytM13h3tZuCfJmBy9P0s04o+pK5w+DX8iQ+OKld8Ey62xJn4lDfdnQV3WhsPksNiyWtdpCYzl/RmhJ+ZEjSbp48PtBXuGK56bpee6305Pp76L3fW1/kPflfz6YpK/bFw6R/+Xb9iTp7b2Vy9xZWJYrZToMWoYAAEDUqAwBAICoURkCAABRoyNEja31+vuszfT9Oe8NK33g0eEg74NrOpP0z79gqE5XF4f8+Mlge+bQo0l6/PthHL1zw0CSbt8XDtO2vnRKedfRV8tLXJH8vgKSdGw8XUbh4Eg4fX/BpWOUN3aHn9vmnnS7tz28Bzq9NWn8biCZVRqCodSFTGar12evVn1JUD+5qfEkXWzvmWfP6vnf1cnZ8Pvh9+kc7Fx9/S/9bk/5sRMV93PX3RZsz7fUxbFz6XQIDxxLy+uNz9oe7PfTN6W/m3raws+2v72578XmvnoAAIAlojIEAACiRpisCvnzZ4LtYlcaWlnILLfXbUxDYa96fnWzfGLhssPgixPplAb5jjCk46+0bL1rw+PUqEl/JZnIhBROT6TN44XMl7nTC0ENZGYKHp9Kfy5n4XjlFi9M0ZoZyzxVSMNfR8fTUMd3T4wH+x08m87s/mQmr80Ltb10Tzgs92WXLd+MtTHwJ36uNiKZHb3ecvbpJF1s6w7yZge2qxrDE2EI976j3gzlme/t9ZvS+zb7ffenAGhW/juaWbdzUcc4Mh7OEt7hfS4v37Umu3sUaBkCAABRozIEAACiRmUIAABEjT5DZdlIcu7Bv0/S5x78lyCvbW0aU+141kuDvOnN+6o6339+7rZg+9DYTIU9sVC5qXPBdsvGdDhoy7pwiZSWTWnetLf8hnTxtPHNyu9Tke0z0eV1BPGHJEtSt5d3WX/Y12rx5h4WP9Q3kNkv3T6eWQX9sVNpf6LNve01ui7MZTEzF2T7Uc56/VrmG96dle0n5OttS391tWa+t3mvD9t0ofo+Qxf2XUg/0GZy37F06Y7sMlKgZQgAAESOyhAAAIha04TJZjJLVT89lq5Mbpkglz+ad6u3em5+nlGV2aZRN5mGWkb3Hw2v5aGnkvSG8XDYb/fz0+uaHrqh8gkzBjubpihWvGJHb/jC5c9Mkq49zJtqi6u5ODsMeaBj5c/Qu7ErvDd6WtPh2ednVmtQY+XZPxKG8v1n63zhtGpDY2cyM6D70zJkZzce2JoOn5/vub4QF8LEzTz43p+5/b6j4e+mq9d3Z3efU8vpA+lGPrz3Zvu3VnWMZpzSgJYhAAAQNSpDAAAgalSGAABA1Jqmo8rZTDz520dGk/RD3tTskrSmK41l/8Q1G5P09t5wSYH52NUvSNLbdl4T5AXLPWSGXxfz1Z/D1wwx1WZR6Fl/6Z0iklvmr1Z+7HjFvNx0OixehbR/nb/EjTR/GfpD/rsXd7thEcamwykOfvovH0nS//nFu5K0v+zQQiyk/1qt+gk1u2zfnKdH035d2SV0Bjsrf77+klOuNe3jVejdONfuc3p4eNK7jskg78YtaV/NtSu0nyItQwAAIGpUhgAAQNSaJkyWHVrZ2Zo2tV25MVxdfGtf2sxXbWgsuzK9b3rjlVUdA4vjzxKbmUFBHQ1sD/eHqU5mmqP9UE3sIc7WU/uD7cKj30zSs6fCaSnGnkxXMB8/fDJJD167K9iv43W/kKSDsDQaZkd/ONv3oWNp94R3fOL+JP1br70+2G/Q67Yw1Edcc6n85+XIVNh9pOjSvM091c8aX+jKzgB/aY+dmQ62v/z4cJJ+7lB4vJUaGvPRMgQAAKJGZQgAAEStadqfs4tIvnxX2gy32EhKfmIkSdv++8PMzXsWd1Bc0mRm8cQJLxzVyBmRR6bCIJ3fHD1dCPPC0Fh8YTJz6efhjj8Z5E08mY4yGjsYjiw79q109vYzT55N0j1bw9Fj7YTGlo0/+s+1hbMUF9vTLgh9beHfzju39CXpb/zzwSR9/9HRYL/bdq2r6jqsMF0xz+VrtVBw8/OfRNnwkz/DfHaR2lo7PRHOSH5qfCpJ37SlutmuVxJahgAAQNSoDAEAgKhRGQIAAFFr2sB8tf2E/N1aTj6eyUzrgsXtmVmmO/sXeWW4lJPnZyvmNbLP0InMdc14/YQ2ZKY6zll8/YR8Ob+/XUv42bRv3pakC5NhP5DOgXSo/eArr0rS697wc8F+fklkuz74n/xyz669GmT75uSmzqV5mdnDp7dcW/E445NpKV13Y7qaeWvm4by1yulNWg7cF15nZ9pfaWbzviCvvr1hVrb5phvZ0pP+Ss/2gfT7E9Xivjk0MhFs//i+zUs/aAPRMgQAAKJGZQgAAERtRYfJglmhswuidqQLv9n0+SCv5fT+JD37dBoaK2aa8/Nb0llvi72XB3kxN8PWgx/qOJ0JR3W3NS405o3q18GzYbPv3sGuJJ2d2iH2WaddziuzdduDvLaeNMScHwiHzPfe+PwkbZddl6Rn+yo3sU/Ohs39/uzfWLjsMPWZwcsr7Bn660dOB9uj3tDqd/7Q3iS9b304rLrqLg1dfeF2IT1+9hnv2rqE+WVXbai1Dd3h9+iGTdWVSW5qPNj2p29oJJ4qAAAgalSGAABA1KgMAQCAqK3oPkO+3OGHgu3i0XRa/6D/gqTZtnTV+tnj6TTxxfNh3LlzKB1OTx+h+vKHdX7ryEiQNz6V9iH64T1hH5Mr14UrZdfagZF0mHF7S/i3Qae3nV2KIHZ+n41CZukM86elWBP2J6p2dewzk+lq3I2cbiF2f/9EumTKr388XLLoxTelZfu8bWkfzvmGfs9nZv3uYDt/7lSSLtJHaMW5anBxS26s1P5ePOEBAEDUqAwBAICoregwWdCkvvmKIK9lYEu64cKht/52y/Z0yKcmzwW7FZhletn4o6GzK8B/8cF0VuKxyXDY/euuT8t515qlr1x9PDOs3w+PPmND2Oxb76GpTc0Ljbl8WJ4u7+dVV2anvbCYJHUxfL4h7jsWdiX4v//wWJLed83GIO89t6ZhrcWGxnzZ70qhd2OFPbESrO+qPnztfzucrcx7e2VeFQAAwDKhMgQAAKJGZQgAAERtRfcZ8hW61827XY3c5GiwXezoq7Bn9YqZMfmz3gvZJRxQ8uqrNgTbjx0bS9L3PnEqyFvXk/Yj+Imr059b2xF+df0uJjOZLmRj02l/lM7M8PmNXU1zC6ws3srnucmxIKu4iL542XKptg9KfiKcpsHl0yV3skN4mT5jbpPeWjnD58MV7X/6lnTJoh/ZHU6LUO/n20rtW7KS+UtY5TL3hp3zllNpzfTP6t+apquc/mIhmuHe49sGAACiRmUIAABELaoYQS3CYpI07TUrn8vEZPym49UeJsu+O39l6flmjN3UHX7t3u0N0f3i4+HK2PcdTJt9P+O9vnddOAx+oDMNjwx2tQZ5/vlWe5ksF5v1wimZGahdbuGPlc6W6ssl533PbCazmrl534tiOI1C9jpR4ockb94ePiO7q5ziYL7Sa4YQyUqW/Wz9zzPb9SM/djxJF55+LMib8VZtaNt9bXjMgaElXeNqQMsQAACIGpUhAAAQNSpDAAAgagTRqzAyFfYL8pcKaM30QWnNxdMn5aK+AN4yDPnxk0FWoWd9du/EWm9V8tftC/f74d1rk/T4dFoOazvDqeB7WL5hyRbS76PY3lPPS6lasS2zcnZLOmR4MX2XYldtH6Es+gUtH/8+NZf55LP95Dwt6zanP9cSDq23YiG7e3T4DQIAAKJGZQgAAESNduQK/BHzhUxTpPO2Cc+k/FWni5lZTP0hn9khzsWO3jmPIYUhND+9ksw39BW1N9+0Dagtf3ZqP0wtSUfH0+kV+tvTe3pbbzi1xbj3MPWnJZGkvrb0+cm0FwtXbA/DxDnv+dmyKRwu7yz9fIuZFRz8Z3CwwnwNrrFZ8JscAABEjcoQAACIGpUhAAAQNfoMVeB3BVqpfVVWsuyw5kLvxiSdmxoP8vIjR5O0nT0a5BXHznob6fDPXH8Y81b/pvRcPYPhtXh9TOqxEvZqiatn3wc9OOKT7dNzdjK957528GyQ9wdffDRJDx8ZS9JXX7sp2G/HYHr/vXB3eG/esqM/SdNnaBEyz9nZDXuTtM1OBnmupaPiYVbLM2wpaBkCAABRozIEAACiRpgMyy47e3GwvXZHVcdgvtT6W2zTedH7walM2KXoTUux2NmOUT/ZUNWm7vRXxGuuCkNcr9yThqr96Ufa5pmFn1DYwi32PpwvLFZvo5lpGJ4eTadh2NobTp/S3175OfCDU1NJ+sp17TW6urnxNAIAAFGjMgQAAKJGZQgAAESNPkMA5uQvo/C5R04FeQOd6ZILw+eng7yOlvRvrLsfGw7yxibTVbXzXt+SF1+xPtjvxi3pkOt69xXA4nS2BOunN+w6MLczk2HPyoMj6X2aXWJq/9mJJO0vrbJrbbj0zVPefr/xt98P8m55Rjqlwm17wv5l/++e/Un6bS++PMi7ZrAzSWe7m+1du3z3Pi1DAAAgalSGAABA1AiTAZhTjzf0/SWXDQR5/grmV68PV85++OS5JH35hnAahbPnZ5J0mxdOu3IwPMbUbHr8J86GYbj1XeljqyczPH+eUd1AVAYyKycMdHRW2FO6YVNXxTzfzv40PP6St95U9bXc+Jprqt7Xt5z3My1DAAAgalSGAABA1KgMAQCAqNFnCMAl+csySJK6595Pkob61iTpl+9aU3nHGpiYDYcIh8O9AaA6tAwBAICoURkCAABRM+eqXxPXzE5KOlC/y8El7HDOrb/0bpdGWTZczcpSojxXAO7N1YOyXF2qKs8FVYYAAABWG8JkAAAgalSGAABA1JqmMmRmm8zsk2b2hJk9bGafN7O9ZrbTzL5Xp3O+ycxOmtkD5X//sR7niU0jyrJ83n9dPt9DZvbxep0nNg26N3/Xuy8fNbOz9ThPbBpUlkNm9o9mdr+ZfcfMXlGP88SoQeW5w8y+Ui7LO81sWz3OU2tNMc+QmZmkv5L0Uefca8uvXS9po6RDdT79nzvnfrbO54hGo8rSzPZI+kVJNzvnzpjZhnqdKyaNKk/n3Nu9a/g5Sc+s17li0cDn7Lslfco59wdmdrWkz0vaWcfzRaGB5fnbkv7EOfdRM3uppDskvaGO56uJZmkZeomkGefcH154wTn3gHPubn+ncm33bjO7r/zv+eXXN5vZXeW/Ir9nZi80s7yZfaS8/V0ze7uwHBpVlm+R9H7n3JnyOU/U8T3GZCXcm6+T9Imav7P4NKosnaS+crpf0pE6vb/YNKo8r5b0lXL6HyW9qk7vr6aaomVI0j5J365ivxOSbnPOTZZbAj4h6UZJr5f0Refcr5tZXlKXpOslbXXO7ZMkM6s0Ve5PmtmLJD0q6e3OuXq3RK12jSrLveW8eyTlJb3XOff3S343aOS9KTPbIekySV9d2tuAGleW75X0pXILX7ekW5f8TiA1rjwflPSTkv6PpFdL6jWzdc65U0t+R3XULJWharVK+v1yU2BB5V+Akr4p6cNm1irps865B8zsSUmXm9nvSfo7SV+a43ifk/QJ59yUmf2MpI9Kemnd3wWk2pdli6Q9km6RtE3S3Wa2zzlHX5PlUevyvOC1kv7SOVeo47UjVOuyfJ2kjzjn/reZPU/Sx8r3ZrH+bwWqfXm+s3y8N0m6S9JhSbN1fg9L1ixhsockPauK/d4u6bik61Sq2bZJknPuLkkvUqlQPmZmbyyHS66TdKekt0r6UPZgzrlTzrmp8uYHq7wGzK8hZSnpaUl/7Zybcc49JekRlSpHWJpGlecFrxUhslppVFm+WdKnysf4uqQOSYNLeSOQ1Ljfm0eccz/hnHumpF8uvzay5HdTZ81SGfqqpHYze8uFF8zs2Wb24sx+/ZKOlv+ieINK4ZALTeknnHMflPRHkm4ws0FJOefcpyW9R9IN2ZOa2WZv83ZJ36/he4pVQ8pS0mdViqGrvP9eSU/W9J3FqVHlKTO7QtKApK/X+D3FqlFleVDSy8rHuEqlytDJmr6zODXq9+agmV2oW/yipA/X+H3VRVOEyZxzzsxeLel9ZvYLkiYl7Zf0XzK7fkDSp83sX6nUcetc+fVbJL3LzGYkjUt6o6Stkv44U2hZbzOz21Vq4jst6U21ek+xamBZflHSD5nZwyo1Bb9rpcewm0EDy1MqhVc+6ZhGvyYaWJb/VdIHy51xnaQ3UaZL18DyvEXSHWbmVAqTvbVW76meWI4DAABErVnCZAAAAHVBZQgAAESNyhAAAIgalSEAABA1KkMAACBqVIYAAEDUqAwBAICoURkCAABR+/8BL+JEqDg/ZwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 720x360 with 10 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "coef = clf.coef_.copy()\n",
    "# 绘制图形，一共10个，每行5个\n",
    "plt.figure(figsize=(10, 5))\n",
    "# 取系数中绝对值最大的数\n",
    "scale = np.abs(coef).max()\n",
    "for i in range(10):\n",
    "    l1_plot = plt.subplot(2, 5, i + 1)\n",
    "    l1_plot.imshow(coef[i].reshape(28, 28), interpolation='nearest',\n",
    "                   cmap=plt.cm.RdBu, vmin=-scale, vmax=scale)\n",
    "    # 设置坐标轴，不显示坐标\n",
    "    l1_plot.set_xticks(())\n",
    "    l1_plot.set_yticks(())\n",
    "    l1_plot.set_xlabel('Class %i' % i)\n",
    "plt.suptitle('Classification vector for...')\n",
    "\n",
    "run_time = time.process_time() - t0\n",
    "# 打印运行时间\n",
    "# print('Example run in %.3f s' % run_time)\n",
    "print('本例运行时间：%.3f s' % run_time)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "上图展示的是分类之后的模型，根据模型，能够更好的判断手写数字是属于哪一类的。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.2"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": false,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
